3 


Parent Choice Versus Attendance 

Area Assignment to Schools: 
Does Macpiet-Based School Choice 
Affect NAEP Scores? 

Douglas A. Archbald & David Kaplan 

University of Delaware 


Abstract: I nter- and intra-district publicschool choice, vouchers, 
tuition tax credits and other forms of school choice have been 
advocated for decades, in large part on grounds that the market 
forces engendered will improve public education. There are 
many studies of school choice policies and programs and a large 
theoretical literature on school choice, but thus far no studies 
have used a large national sample and common metricto perform 
a multi-level, multi-district analysis of relationships between 
school choice pol icy and student achievement. This study I i nks a 
national sampleof NAEP student achievement data, with district 
level information on magnet-based school choice policy, andwith 
demographic data from the U.S. Census. Using three-level 
hierarchical linear model ingwefind substantial effects of school 
and district demographic variables on student achievement, but, 
after adjusting for multi-level demographic characteristics, we 
find only small differences in student achievement in school 
districts with magnet schools and school choice policies as 
compared with districts with attendance area based student 
assignment, and no magnet schools. NAEP achievement scores 
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are marginally lower in the sample of students within districts 
reporting magnet schools and associated school choice policies. 

Introduction 

School choice in a variety of forms has expanded nationally and 
internationally, in large part on the belief that market forces can improve 
publ iceducati on (Peterson 2001;T eel ken, 1999; Van Zanten, 1996; Whitty, 
Power, & H al pi n, 1998). I n theU nited States, magnet schools werethefi rst 
school choice policy widely implemented in public education (Blank, 
Dentler, B altzel I & Chabotar, 1983; Levi ne& H avighurst, 1977). Research- 
ers have studied magnet schools to learn whether this approach toschool 
choiceaffects achievement, but in almost all cases studies have focused on 
justasingleorafewdistricts.Thisstudy merges existing national datasets 
to perform a multi-level, multi-district analysis. It charts new ground by 
linking student achievement data from the National Assessment of 
Education Progress (NAEP) with district level policy information on school 
choice and demographic data from the U .S. Census. 

Of the many different forms of school choice in existence (or proposed), 
magnet schools implement a regulated form of school choice— neither 
pr i vate school s nor ch a rter school s a re i n vol ved , a nd pa rents' ch oi ces may 
be restricted in the interests of racial desegregation of schools. Nonethe- 
less, thepresenceof magnet schools i n a district sti 1 1 produces a great deal 
of parental choice, raising questions about whether school districts with 
magnet- based school choi ce mi ght h a ve h i gh er ach i evement thandistricts 
lacking school choice. This is the question we examine. 

Theory and Research on Magnet Schools, 

School C hoice, and Student Achievement 

The Growth of Magnet Schools and School Choice 

Whi lecharter schools and voucher programs have been getti ng more 
attention recently, magnet schools remain one of the most common 
approaches in the U.S. (CER, 1998; Steel & Levine, 1994). The "magnet 
school boom" (Warren, 1978)ofthelate70sandcontinuingintothe80s 
was the precursor of the widespread adoption of public school choice 
policies we see today. Since 1976 when only 14 districts had magnet 
schools (Blank, Dentler, Baltzell & Chabotar, 1983), the number of 
districts with magnet schools has grown rapidly along with enrollment in 
magn et sch ool s ( B I a n k , 1989 ) . B y 1990 est i mates wer e t h at a I most h a I f of 
urban districts had magnet schools (Steel & Levine, 1994). 
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When a district implements a magnet-based school choice policy, it 
designates a portion of the schools in the district as magnet schools. 
M agnet school s are open to en rol I ment by choi ce on a di str i ct-wi de basi s, 
subject to racial balance guidelines (sometimes regions within districts 
are used if thedistrict is geographically large). Magnet schools typically 
have special features, programs, or instructional methods that distin- 
guish them from "traditional" schools. M agnet schools may offer distinc- 
tive instructional approaches such asMontessori or Waldorf; curricular 
emphases such as art, math, science, technology, or foreign language 
i mmersi on; or disci pi inary cl i mates i nvol vi ng above average regi menta- 
tion and required uniforms. 1 1 n addition, magnet schools sometimes offer 
special features like smaller class sizes, instructional aides, or newer 
facilities, features made possible by federal funds from the Magnet 
Schools Assistance Program that has supported magnet schools' racial 
integration function since the mid-70s. Districts with magnet schools 
often extend choices to cover other schools i n thedistrict as wel I . Parents 
may choose and enrol I a child in any school as long as the choice does not 
increase racial desegregation in thedistrict. A child in aschool wherehis/ 
her race is in the majority can transfer to any school where his/her race 
is in the minority. 

Theory and Research on Magnet-based School Choice 

A large academicand practitioner-oriented literaturesupportedthe 
growth of magnet schools and publicschool choice. M agnet-based school 
choice was supported because it hel ped vol untary raci a I i ntegrati on (e.g., 
Greeley, 1987; Levine & Havighurst, 1977; Rossell, 1990), expanded 
educational alternatives for parents, and stimulated competition in 
publicschools(Barr, 1982; Blank, 1989; Clinchy& Cody, 1978; Finn, 1985; 
Finn, 1987;Kolderie, 1985; Nathan, 1989; Manley-Casi mir, 1982; Raywid, 
1985; Toch, Linnon, & Cooper, 1991, May 27). 

Researchers and policymakers have long been interested in achieve- 
ment effects associ ated with magnet schools (e.g., Arch bald, 1995; Blank, 
1989; B lank & Arch bald, 1992;Gamoran, 1996;Goldhaber, 1999; Larson, 
Witte, Staib, & Powell, 1990). Two inter-related arguments have sup- 
ported the growth of magnet schools, both suggesting achievement 
benefits. Oneargument isthattheexpansi on of educational alternatives 
improves the fit between a school district's educational program and its 
cl i entel e's preferences. M i chael son ( 1981) has referred toth i s as "al I ocati ve 
efficiency." It reflectstheassumptionthat notall parents want thesame 
thing, and so rather than force all parents to accept the same kind of 
school program, school systems should have differentiation among 
schools reflecting different educational philosophies, pedagogies, and 
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curriculumspecializations. I ntheory, magnet schoolsdothis. Whilethey 
may not be for everyone, they make it possible for a school district to 
accommodate more parents with varied preferences than would other- 
wise be possible with a "one size fits all" philosophy. This viewpoint is 
prominent in the literature supporting magnet schools (e.g., Fliegel, 
1993; Clewell & J oy, 1993; Nathan, 1989; Raywid, 1985). 

The second argument reflects free market principles. Advocates of 
magnet schoolshavearguedthattheyfoster healthy competition in school 
systems. School sthat have long been assured a desi gnated attendance area 
clientele, are in a more competitive position when a school system 
i mpl ements magnet-based school choi ce pol i ci es. C h i I dren from the nei gh- 
borhood are no I onger guaranteed to a school . When the market i s opened 
up, asthetheory goes, schools will try harder tosatisfy parents. Observers 
di sagreewhether "tryi ng harder" necessari I y results i n i mprovi ng curricu- 
lum, instruction, and school discipline in ways that raise academic 
achievement. Still, these two inter-related arguments have been very 
i nf I uent i al i n expandi ng t he adopti on of pu bl i cschool choi ce, and sti 1 1 propel 
the growth of charter schools and voucher programs. 

Ideally, one would want to compare achievement in a sample of 
districts with magnet-based school choiceovertimetoa matched sample 
of districts without these policies, but this has not been donefor a variety 
of reasons, not least of which are the methodological difficulties such a 
study would pose. One research method compares magnet students' 
achievement to the achievement of other students in the district not in 
magnet schools. These comparisons typically find magnet school stu- 
dents have higher test scores.For instance, Archbald (1995) compared 
ach i evement scores of el ementary magnet students, nei gh bor hood school 
students, and students who had "choiced out" of a neighborhood school, 
but not to a magnet. The results showed a statistically significant effect 
of magnet school enrollment, although theeffect was fairlysmall (about3 
NCE pointson average). Larson etal.(1990)comparedhigh school magnet 
students to a sample of non magnet students on mathematics and science 
tests. Since it was known from unadjusted cross-sectional comparisons 
that the magnet students had higher test scores (on average) and that 
several of the magnet programs were selective in their admissions, 3 the 
comparison group was randomly sampled from col lege-prep courses i n the 
nonmagnet high schools. Their analyses found modestly, statistically 
significantly, higher mean test scores in the magnet programs, even when 
adjusting for prior eighth grade achievement. 

An important methodological limitation of the above magnet- 
nonmagnet comparisons is selection bias. Ideally, students should be 
randomly assigned to magnet and nonmagnet schools, but, of course, 
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students are in their schools based on their own and their parents' 
decisions, and so the two groups are possibly different on unmeasured 
covariates of the characteristic that disti nguishes the choosers from the 
nonchoosers. If these covariates are associated with student achieve- 
ment, then this confounds the magnet-non magnet comparisons. Studies 
of differences between choosi ngand nonchoosi ng parents i n school choice 
systems find that parents who take advantage of school choice opportu- 
nities often differ in ways that would predict higher achievement among 
their children (Arch bald, 1996; Levine, 1975;Smrekar & Goldring, 1999; 
Wells, 1991; Witte, 1998). 4 

Gamoran(1996)conductedalargenational-samplestudy comparing 
magnet and nonmagnet schools usingtheNational Educational Longitu- 
di nal Survey (N CE S, 1988). Gamoran focused just on urban districtsand 
compared schools that identified themselves as a magnet schools_with 
schools that were not identified with any specialty designation. Among 
other thi ngs, Gamoran found students who attended magnet schools had 
higher reading and social studies achievement scores, after controlling 
for pre-existi ng differences among students. 5 

The above studies, done in the 90s represent the best research on 
magnet school achievement effects. There have been a few other, older 
studies of magnet school systems as reviewed in Blank and Archbald 
(1992), but these studies had such serious methodological shortcomings, 
usually no comparison groups or demographic adjustments, that the 
authors con cl uded "I i ttle i s known about the effects of magnet school s on 
the quality of education in our schools" (p. 85). 

Research on Other Forms of School Choice 

The magnet-based form of school choice examined in this study is 
among the more regulated approaches. Many researchers and school 
choice advocates believe the monopolisticstructureof public education 
must beforcedtochangethrough stronger free market mechanisms, and 
areskepti cal that the regul ated magnet-based choi ce pol i ci es i mpl ement 
market forces sufficient to cause under-performi ng systems to i mprove 
(Chubb & Moe, 1990; Erickson, 1982; Everhart, 1982; Friedman & 
Friedman, 1981; Harmer, 1994; Hoxby, 1996; Lieberman, 1986; 
Michaelson, 1981; Moe, 2001; Peterson, 1990; Peterson 2001).Twokey 
differences separate magnet-based school choice from the less regul ated 
charter school and voucher policies. One is the racial balance goals 
regulating admissions to magnet schools and other participating choice 
schools in systems with magnets (choice is allowed, but restricted to 
inhibit resegregation); the other difference is that the magnet-based 
approach to choice is still "within the system"— that is, the district 
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bureaucracy and teacher association regulations still exercise substan- 
tial control over the management and operation of all schools. 

Charter school sand voucher eligi bleprivateschoolsareconsiderably 
more i ndependent and thus, in theory, create a more open marketplace 
of school options. Charter schools receive state funding in proportion to 
their enrollment to fund operating costs, but are essentially indepen- 
dently managed organizations. M any charter schools are even operated 
through private enterprise arrangements (GAO, 2002). In voucher 
programs, it is the private schools that participate in the market and 
becausetuition vouchers greatly expand accesstoprivateschools, they— 
again, in theory— create greater inter and intra sector competition. 

It is beyondthescope of this paper to review all the research on these 
differ entforms of school choiceotherthan notethattheexistingresearch 
has not converged toward definitive conclusions (GAO, 2002; Gill, 2000; 
Goldhaber, 1999). One body of research stems from evaluations of a 
handful of pioneering voucher programs in the U.S.— in Milwaukee, 
Dayton (Ohio), New York City, and Washington, D.C. (Green, Peterson 
& Du, 1998; Howell, Wolf, Peterson, & Campbell, 2000; Rouse, 1998; 
Witte, 1998). 6 These evaluations have been methodologically sophisti- 
cated, even incorporating random assignment principles. Green et al. 
(1998) and Howell et al. (2000) discovered significant achievement 
benefits accruing to the children who received vouchers and attended 
private schools; however, Rouse (1998) and Witte (1998), found much 
smaller and in some analyses no positive effects. One of the main 
methodological shortcomings of the studies finding significant positive 
effects is differential attrition of subjects from the different conditions 
over time. A key concern was the loss of apparently less academically 
competent students from the private school condition— that is, it ap- 
peared that there were a number of students awarded vouchers who 
attended the privateschools but later returnedtothepublicschools, and 
these on average were I ess academi cal I y successf u I students. 7 Comment- 
ing on the state of evidence on educational vouchers, Levin (1998:3) 
writes, "...both advocates and detractors tend to argue about the conse- 
quences of vouchers more from theoretical and ideological groundsthan 
empirical ones." And on the Milwaukee voucher studies, Levin adds, 
"Even Rouse [Rouse (1998)], the most sophisticated [of the] analyses of 
thesedata, is riddl ed with cauti ons about data gaps and the assumptions 
that are made to address them including the use of instrumental 
variables." Methodological obstacles to definitive research on this ques- 
tion are many, the available sites for research remain limited, and 
ideological interests are strong. 

Research on charter school effects isj ust now emergi ng, but thereare 
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only a few achievement-effects studies usi ng control led charter versus non- 
charter school comparisons (Gil I etal., 2001; Miron & Nelson, 2001; 1999; 
Lin, 2002). A recent review noted the paucity of well designed research 
(despite a voluminous literature on charter schools) and concluded that, 
.evidenceon theacademiceff ectiveness of charter schools is mixed"(Gi 1 1 
et al ., 2001, p.95). According to Miron and Nelson's review (2001, p. 1) 
"Overall , the charter i impact on student achievement appears to be mixed 
or very slightly positive. However, this conclusion istempered bythefact 
that there are, as yet, no systematic studies of charter achievement in 
several states that have large numbers of charter schools." 

Summary Comments on the Theory and Research 

There are many who bel ieve the forces of markets and competition 
can improve public education. This presumption has supported the 
growth and development of magnet schools, inter-district public school 
choice, charter schools, private management of public schools, and 
privateschool voucher programs and policies. Some supporters of school 
choice advocate public-only school choice models. They believe ap- 
proaches like magnet schools or inter-district school choice policies can 
createadequateeducational alternatives and competition among public 
schools. However, as described above, a large literature and advocacy 
grou p devel oped i n t he I ate 80s a nd 90s skept i ca I of t h e prospects of school 
choi ceconfined withi n and managed by thepubliceducati on bureaucracy. 
Illuminating studies have emerged on relationships between school 
choice and academic achievement, but much more research needs to be 
done because of the many varieties of school choice, the multiple levels 
at which effects need to be investigated, and the many challenges in 
measuring achievement outcomes. 

Thisstudy providesa uniquecontri bution i n several ways. 1 1 usesa large 
national sampleof districts, but over comesthe problem of different districts 
usi ngdifferent tests by relyi ngon onecommon assessment: N AE P . Another 
important features of this study is that it is not a withi n-district study 
comparing magnet schoolstononmagnetsfromthesamedistrict, as so often 
has been done; rather it asks and investigates the question: do school 
systems with magnet-based school choice have higher NAEP achievement 
than those without school choice, that is, with attendance zones and 
centrally administered student assignment to schools. 

Method 

A Multi-District Comparative Design 

This study compares student achievement among school districts 


10 


Parent Choice Versus Attendance Area Assignment to Schools 


with and without magnet-based public school choice policies. For its 
school choi ce policy information thisstudy draws on a 1991-92 telephone 
interviewed a nationally representative stratified random sampleof 602 
school districts (Steel & Levine, 1994); for its student achievement data 
this study uses math and readi ng achievement scores from the N ational 
Assessment of Educational Progress (NAEP); demographic information 
to adjust comparisons among the districts comes from the National 
Center for Education Statistic's Common Coreof Data 

Before proceeding to the details of the data, design, and measures, a 
methodological comment on selection bias is warranted. The main 
strength of thisstudy, other than its large sampleand national focus, is 
that its comparative design minimizes the form of selectivity bias that 
affl ids the magnet versus non magnet comparisons discussed earl ier. As 
previously described, when one compares the students in a district’s 
magnet schools to those not in the magnet schools, there are likely 
conf ou nd i n g effects of sel f-sel ect i on : W i t h i n a d i st r i ct , when pa rent s sort 
themselves by choice into magnet schools and nonmagnet schools, 
children of the magnet school parents often have higher test scores 
because of family background variables. A more ideal approach would be 
to compare student achievement between school districts, with and one 
without school choice, but identical i nail other ways (e.g., size, urbanicity, 
student demographics, educational expenditures). If magnet schools 
contribute to higher student achievement, either because they them- 
selves are better schools, or because they engender competition that 
makes all schools better, then thedistrict with school choiceshould have 
higher achievement. Such a difference could reasonably be taken as 
evidence of effects of school choice. 

Comparing school districts attenuates self-selection issues. While 
research indicates that within a school district, magnet school parents 
di ffer from non magnet school parents, there i s I ess reason to bel i evet hat 
at the district level there are large differences in the characteristics of 
parents between school districts with magnet-based school choice policies 
than those without them. While it is conceivable that cities with school 
systems with magn et-based ch oi ce pol i ci es mi gh t d i ff erent i a 1 1 y att r act or 
retain higher SES parents, our methodology controls for key demo- 
graphic variables. Furthermore, if there is self-selection of higher - 
ach i evi ng st udents to magnet di st r i cts, even after demogra ph i c cont rol s 
are applied, then one would expect higher NAEP achievement in the 
magnet school districts. The methodology used in this paper explicitly 
tests district effects and should reveal this effect if it exists. 
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Data, Sample, and Measures 

The data for this study came from three sources: 

I ntervi ew i nformati on on student assi gnment pol i ci es. I n 1991-1992 a 
telephone interview was conducted with district officials in 602 school 
districts to collect information on magnet schools and other types of 
policiesfor school choice and desegregation in a national sampleof school 
districts (Steele& Levine, 1994). The purposeofthetelephone interview 
was not toelicit respondent's opinions. Rather, thepurposewastocol led 
information on types of student assignment policies used in thedistrids 
in the sample (e.g., school choice or other types of student assignment 
policies), whether or not the distrids had magnet schools, how many 
magnet schools distrids had, the types of magnet schools in operation, 
and other distrid policies related to school choice (e.g., transportation 
support, parent information systems) and school desegregation goals. 
Because of the fad-gathering nature of the telephone interview, inter- 
views were conduded with individuals in the school distrid office 
knowledgeable about student assignment policies, magnet programs, 
and school desegregation. Multiple interviews were sometimes con- 
duded, with supplementary information colleded from distrid policy 
documents. Letters were sent to all distrids seleded for the sample in 
advance of the interview explaining the purpose of the study and 
arrangi ng t he i ntervi ews. T he tel ephone i ntervi ews were conduded by 
American I nstitutesfor Research, using a computer-assisted telephone 
interview system. 8 Of the distrids targeted in the sampling frame, 1% 
refused to parti ci pate, and for another 5%, no data were col I eded due to 
diffi culties with schedul ingor makingcontad with an appropriatedistrid 
respondent withi n the ti me frame of the study. 

A stratified sampling procedure was used to develop the distrid 
i ntervi ew sampl e. T he pri mary pu rpose was to maxi mi ze the preci si on of 
estimates of student participation in, the charaderistics of, and the 
prevalence of magnet schools and school choice in the nation's public 
school systems. It was known beforethedesignofthesamplethat magnet 
schools, school choice, and school desegregation policies are quite 
common in larger urban distrids, fai rly common in mid-sizedistrids, and 
not common in tiny distrids with only a few schools. 9 Therefore, to 
increase the precision of estimates, the sample was designed to most 
closely refled this universe of school distrids. The nation's 155 largest 
school distrids were all included in the sample, distrids that were 
overwhelmingly one race were undersampled, and school distrids that 
did not offer the strudural potential for school choice— that is, they did 
not have at least two schools at one grade level— were excl uded from the 
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sample. The final sample of 602 districts was drawn from a universe of 
about 6,400 districts based on 1990 data. 10 As described in more detail 
below, about half of the 602 districts were also part of the NAEP 1992 
sample. Merging the two samples created thefinal samplefor our study. 

Common Coreof Data. The National Center for Education Statistics 
maintains a data set on public education with enrollment, demographic, 
staffing, and financial information (school, district, and state). This data 
set was used for thedemographi c i nfor mati on on theschool s and di stri cts 
i n theschool choice pol icy i nterview sample. 11 To adj ust our comparisons 
amongtheschoolsanddistrictsfor demographiccharacteristics, we used 
theschool level measureof family median income, and the district level 
measure of the percent of children in poverty (the latter being derived 
from 1990 U .S. Census data). 

National Assessment of Educational Progress. Student achievement 
was measured by student scores on the 1992 National Assessment of 
Educational Progress (NAEP) in 4 th grade reading and 4 th and 8 th grade 
mathematics. NAEP 1992 data were used because this was when the 
student assignment policy information was collected. Most magnet 
school programs were i mplemented duri ng the I ate 70s and 80s, spurred 
by the availability of federal funding 12 and growing public and political 
support for voluntary alternatives to mandatory reassignment policies 
for school desegregation. That NAEP was administered in math and 
reading in 1992 at which timethemagnet/publicschool choicesurveywas 
administered is optimal for this study for it was the 1980s that brought 
the widespread implementation of magnet schools and their attendant 
school choice policies. This study assumes if district level effects of 
magnet schools and choice have occurred, these effects should be 
measurable by 1992. 

The use of NAEP achievement scores for this study carries certain 
advantages and disadvantages. NAEP achievement measures areadmin- 
i stered vi a a bal anced i ncompl ete bl ock spi ral i ng desi gn . The advantage 
of such adesign isthat it allows for a largenumber of achievement items 
to be given across thetest population without any given student being 
burdened bytoolargean assessment. A disadvantagefor someresearch 
purposes is the fact that any given student has responded to very few 
items and thus estimates of ability via item responsetheory methodology 
may contain too much error to ignore. Therefore, NAEP provides five 
plausible value estimates for each student conditioned on two sets of 
variables: responses to that portion of the assessment administered to 
thestudent as well as student background characteristics. As pointed out 
i n the 1994 N AE P T echn i cal Report (A 1 1 en, J oh nson , M i si evy, & T homas, 
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1994), the pi ausi bl e val ues are not test scores for i ndi vi dual s i n the usual 
sense, but they do estimate population characteristics. I n the context of 
statistical modeling with plausible val ues, special methods must beuti I ized 
in order to obtain correct estimates and standard errors. These methods 
are incorporated in the HLM software and were util ized in this paper. 

The issue of inferences to relevant populations represents another 
limitation in the use of NAEP for this study. Specifically, NAEP used a 
compl ex, multi-stage probability samplingdesigntoselectpublicand non- 
public schools as well as a probabi I ity sampling of students within those 
schools defined by certain ages or certain grades (Allen, Johnson, 
Mislevy, & Thomas, 1994). Oversampling of non-public schools and 
schools with large minority populations was necessary for representa- 
tiveness. To ensure proper inferences to relevant populations, NAEP 
provides sampling weights associated with school and students that 
rel ate tothe units’ probabi I ity of selection intothesample. However, the 
sampling design of NAEP does not includedistricts. Assuch,thereare no 
wei gh ts associ ated wi t h d i st r i cts to a 1 1 ow i nf eren ces to t h e popu I at i on of 
school districts in the United States, or even the largest districts in the 
United States. Thus, it is important to recognizethattheresults reported 
below are unweighted and do not represent inferences tothe population 
of districts, schools, or students. 

These I i mitations notwithstandi ng, we consider the use of N AE P as 
themeasureof achievement as justified for several reasons. First, NAEP 
is a highly regarded test of major U .S. national policy significance. The 
test is a product of extensive research and development by national 
experts in curriculum, instruction, and measurement and employs broad 
partici pation and review in item development. 13 N AE P uses both multiple 
choice and free response items. Second, uni ike high-stakes tests used in 
many states and di stri cts, teachers have no incenti veto "teach to" NAEP; 
schoolsanddistricts, sincetheyarenot identified, havelittleincentiveto 
"look good" on NAEP. Therefore, there is no reason to suspect test score 
inflation biases in the sample. Third, whilethere is likely to be variation 
amongthedistrictsinthesampleinthedegreetowhich their reading and 
mathematics curriculum align with NAE P's content specifications, there 
arenoa priori reasons to assume systematic differences creating biases 
affecting the adjusted comparisons of the models estimated in our 
analyses. Fourth, NAEP uses an extremely large national sample of 
students. N oother assessment measure meeti ng al I of thecriteri a above, 
and alsoprovidi nga large sampleisavailabletoconductthekind of policy 
study reported here. 

Final sample. Ourfinal sample, then, consists of a large number of 
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racially diverse, multi-school districts, about half of which have school 
choice policies of some form. The sample intentionally over-represents 
larger districts because it is in these systems where magnet-based school 
choice policies are predominantly found. Thedistrictsfor our final sample 
are those with both student assignment policy information and student/ 
school NAEP scores. E ven though this study uses just a subset of NAEP 
data— only thosedistrictsthatwerein both the 1992 NAEPsampleandthe 
602 district school choi ce study sample— the merged sample still is large: 
morethan 30,000 students, 1,000 schools, and 300 districts. 

Coding of thesample About 300 districts had both student assign- 
ment policy information (e.g., information on magnet schools and 
associated programs and policies) and NAEP scores, which lead to a 
sampleof about 30, 000 students i n 1, 000 schools. With the(district policy) 
i nterview i nformati on, it was possi bletocodethestudents and schools i n 
thesample according to their district student assignment policies. The 
main question on the interview form used for this coding asked whether 
the district had magnet schools. Magnet schools require school choice 
policies. A specific definition of magnet schools was read to the respon- 
dent to insure reliable responses tothis question. 14 Additional questions 
asked about transportation and parent information support. Other 
questions asked whether thedistrict had a "majority to minority" form of 
choi ce pol i cy— an approach that does not specif i cal I y use magnet school s, 
that al lows parents some school choices, and that regulates enrol I ments 
in the interest of racial desegregation; and whether the district used 
neighborhood school s-based student assignment or rezoning and busing 
policies for school desegregation. 

The pol icy i nformati on from thetelephonei nterviews was used tocode 
thestudent/school data sampleand conduct several comparativeanalyses: 

High Mag v. LowMagv. Other. Thestudent/school data were coded 
as to whether they operated in a "high mag," "low mag," or "other" (no 
school choice) environment. Both "high mag" and "low mag" students/ 
schools were in a school choice pol icy environment, but a distinction was 
madesothat two dichotomous variables were created, thus permitting a 
more robust test of the magnet/school choice variable. 

The "high mag" versus "low mag" distinction was created because- 
as the market theory goes— the more school options are available with 
supportingtransportation and information systems, themorethesystem 
approximates an education marketplace with presumed productivity 
benefits stemming from competition. In theory, school systems that 
provide a large supply of magnet schools and support parental/student 
access with well developed transportation and information systems 
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create a more open and robust market. For instance, if a school system 
has 30 schools and only 2 are avai I able as choice schools, and there is no 
publictransportation availableforthese schools, and littleadvertising- 
it is hard to argue that this system resembles a market, although 
technically such a system has school choice. On the other, a market is 
produced, arguably, when there are large numbers of accessible choice 
schools supported by transportation and parental information systems. 
The interview information on district school choice systems was used to 
categorize systems as "H igh Mag" in which morethan20%of their schools 
were magnet schools, free bus transportation was provided to all schools 
in the system, and choice was supported with parental information 
systems using at I east threedifferent media (e.g., a district radiostation, 
newsletters, and magnet school open houses). Districts that did not 
achieve all three of these characteristics were classified as "Low Mag." 
(Note the terms "high" and "low" are arbitrary.) The use of two dichoto- 
mous variables (rather than just one) lessens thechancethat an absence 
of observed effects might beduetosome of thestudents/schoolsoperating 
in an environment classified as "school choice," but in reality offering 
relatively fewer options. 

Neighborhood v. Other. The student/school sample was coded as to 
whether the policies were of the "neighborhood schools" type or "other." 
A "neighborhood schools" policy refers to assigning children to a school 
based on their residence within an attendance area boundary which is 
drawn to encompass the neighborhoods surrounding and closest to the 
school. "Other," in this classification refers to either choice policies, 
which could include magnet schools, or various types of mandatory 
reassignment for school desegregation purposes. 

"Other" C hoi cev. MagnetsOr NoSchool Choice. The student/school 
sample was coded as to whether the policies permitted selected types of 
school choice through forms other than those associated with magnet 
schoolsordid not. The "other" category exists becausethesurvey sought 
to identify kinds of policies that technically fall under the rubric of 
"choi ce, "though in fact these areoften fairly limited for ms of choice, such 
as allowing transfers among schools through special request. Some 
districtshavepre-existingarrangementswithoneor several neighboring 
districtsand allow limited numbers of student transfers. Somedistricts 
allow transfers among schools where the transfer improves racial 
balance— that is, parents may select a school outsidetheir neighborhood 
ifthechosen school is predominantly composed of students of a different 
race. These modest forms of choice constitute the "other" category. 

Weshould emphasizethat the primary contrast of interest isthefi rst 
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descr i bed a bove: "h i gh mag v . I ow mag v. ot h er . "T h i s i s t he st r on gest test 
ofthemagnet-based school choice model. I nthe"high mag"and"lowmag" 
districts large percentages of parents inthedistricthavegenuinechoices 
amongavarietyofschoolswithdistinctivethemesandprogramsandthe 
schools in thesedistricts operate in an environment that places schools 
i nto more of a competitive rel ati onshi p than they woul d otherwise be i n 
atraditional "nonchoice."The"other"category, towhich "high mag"and 
"low mag" districts are compared consists of districts in which students' 
placement in schools is by central administrativeassignment, which does 
not give school choices to parents. 

However, all threedifferent types of contrasts described above have 
been examined. First, it was possible to do so based on the information 
from the telephone survey; second, these different student assignment 
pol i cy options are i ncl uded i n discourse and debates about the pros and 
cons of different forms of student assignment policy, including all the 
different forms of public school choice. Therefore it is worth examining 
if they appear to be associated with any differences in achievement as 
measured by NAEP. 

Additional sampl einformati on. Tables 1 through 4 provide informa- 
tion on the sample. These figures are from the telephone survey (district 
magnet status), fromtheCCD for theschool year 1991-1992 (%freel unch; 
% black), and from 1990 U.S. census (adult education; family income). 
Table 1 compares the demographics of the districts with and without 
magnet schools. The main difference in these two categories of districts 
isthatthedistrictswith magnet schools (constituting approxi matelyone 
third of the districts) are on average larger and have higher percentages 
of minority and low-income students. As other studies have shown, 
magnet schools are far more likely to be in urban districts with high 
minoritypopulations(Blank, Dentler, Baltzell &Chabotar, 1983;Steel & 
Levine, 1994)— not surprising because magnet schools are associated 
with racial desegregation programs. The districts with magnet school/ 
public choice policies have on average a little more than twice as many 
schools and students: a mean of 79 schools in magnet districts and 36 in 
nonmagnet districts; and a mean of 49.5 thousand students in magnet 
districts versus 23 thousand in nonmagnet districts. The percentage of 
black students and free-lunch eligible students is higher in magnet 
districts by, respectively, about 10 percent and 14 percent. The adult 
populations living in the school districts with magnet schools are also 
demographically of somewhat lower socio-economic status than the 
populations in districts without magnet school /public choice policies. 
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Table 1 . Demographics of "Magnet" versus "Nonmagnet" Districts 

District 
M agnet 
Status 

% F ree 
Lunch 

% Black 

% Poverty 

Adult 

Education 

Family 
1 ncome 

No 

32.4 

14.6 

17.3 

12.6 

40308 

Yes 

46.7 

24.7 

23.8 

12.6 

38403 

Total 

37.3 

18.1 

19.6 

12.6 

39646 

Table 2. Demographics of "Neighborhood Schools" versus 


"Not Neighborhood Schools" Districts 



Neighborhood 

% F ree 

% Black 

% Poverty 

Adult 

Family 

Schools? 

Lunch 



Education 

1 ncome 

No 

37.3 

20.7 

21.6 

12.6 

38290 

Yes 

29.7 

13.3 

15.9 

12.6 

42153 

Total 

37.3 

18.1 

19.6 

12.6 

39646 


Table 3. Demographics of "Other Choice" Districts 


Other 

Choice? 

% F ree 
Lunch 

% Black 

% Poverty 

Adult 

Education 

Family 
1 ncome 

No 

37.2 

19.6 

19.5 

12.6 

40026 

Yes 

36.3 

14.5 

19.2 

12.7 

38980 

Total 

36.9 

18.2 

19.4 

12.7 

39813 

Table4. Demographics of Parent Sampleand NAEP Subsample 

"Parent" 

Sample 

% F ree 
Lunch 

% Black 

% Child 
Poverty 

Adult 

Education 

Family 
1 ncome 

All District 

34.5 

14.8 

18.5 

12.7 

40507 

NAEP 

subsample 

36.0 

18.1 

19.7 

12.6 

39671 


Table 2 provides comparative demographic information for the 
di stri cts cl assif i ed as ut i I i zi ng a nei gh borhood school s pol i cy as compared 
to all other districts. Districts with neighborhood schools tend to be 
smaller by about 17.5 thousand students on average, and have lower 
percentages of black students and students eligible for free lunch, by 
about 8 percent and 12 percent, respectively. This, presumably, reflects 
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thefact that larger districts with higher percentages of black studentsare 
the ones that either through court or school board directives, have 
developed alternative student assignment policies in an effort to reduce 
racial isolation in schools. 

Table 3 shows relatively similar characteristics of the districts 
classified as "other choice" as compared with those classified as "noschool 
choice." As described earlier, from either a theoretical or practical 
perspectivethiscontrast istheleast interestingofthethree. Presumably 
because each of these two categories represents something of a mixture 
of student assignment policy types, there appear not to be demographic 
covariates of the policy categories. 

Asdescri bed above, the sample for this analysis is created out of the 
common cases from two different samples. It should be noted that while 
the original telephone survey sample (n=602) was a national random 
sample, the extent to which thesamplefor this analysis is truly random 
is unclear. There is no a priori reason to assumethesampleon which our 
analyses are based is not random or nearly random, for it is created out 
of two different samples, both of which are stratified random samples. 
Comparing the analysis sample to the original "parent" sample of 602 
revea I s f ew notewort h y d i ff eren ces as show n i n T a bl e 4. T h e mea ns of t h e 
demographiccharact eristics of theanalysissampleascompared with the 
parent sample in most cases differ by just a few percentage points; just 
as important, there is no pattern to the differences, suggesting these 
differences where they occur are random. 15 

Analytical Methods 

The same general analytic strategy is employed for each of the 
different types of comparisons. Becauseofthenestedstructureofthedata 
(with students nested in schools that are, in turn, nested in districts) we 
employ three-level hierarchical linear modeling, using the software 
program H L M , to correctl y account for the nested structure of the data. 
(Bryk, Raudenbush, & Congdon, 1994). Weextend previous research by 
considering the effects of school choice as a three level model. By 
specifying a three-level model with choice modeled at the district level, 
we address the multi level nature of the data in a way that has not been 
done before in school choice research, as well as address the issue of 
selection bas-arguingthat selection biasismuch moreof a problem when 
choice is measured at the school level than at the district level. 

Ouranalytic approach sped f i es a ser i es of steps. T o begi n , we assess 
the proportion of variance in student achievement accounted for by 
differ encesamongstudents, schools and by districts. This is accomplished 
by decomposing the total variation in student achievement across 
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students, schools, and districts, without adding predictors at any level. 
Next, we add the choice comparison at the district level in a three-level 
model and recompute the variance accounted for, conditional on the 
choice com pari son. Next, we add statistical controls at each level, one at 
atime. Atthestudent level, we add level of parental education (PARED) 
as a proxy for student socio-economic status, at the school level we add 
school median income (SCHINC), and at the district level we add the 
percent of students living below poverty (POVERTY). Our interest is in 
determining the effect of school choice on student achievement after 
control ling for PARED, SCH INC, and POVERTY. Therefore, we grand- 
mean center PARED, SCH I NC, and POVERTY yielding results that can 
be interpreted as adjusted achievement means, adjustedfor the statisti- 
cal controls at each level. 16 For each step we report the reduction in the 
variance at each level as well as changes in the mean difference in 
adjusted student achievement as a function of thechoice policy. A more 
complete discussion of the statistical model employed in this study is 
given in Appendix 1. 

Our statistical comparisons (e.g., "magnet" v. "nonmagnet") were 
adjusted using these demographic variables for both theoretical and 
practical reasons. The main theoretical reason is that, whether measured 
at the individual level or aggregate levels, student achievement has a 
strongcorrelationwithfamilysocio-economicbackgroundvariables(J encks, 
1972; Kerckhoff, 1996). Theoretically, the effects on productivity of new 
pol icies and reform i nitiati ves should operate i ndependently of the demo- 
graphic backgrounds of the students (this assumes that the policies/ 
initiatives are not of a type intended specifically to be more effective for 
certain demographictypes of students.) Theories about effects on produc- 
tivity of market forces engendered by school choice do not make distinc- 
ti ons about differ enti al effecti veness rel ated tostudent background. Thus, 
it is reason able to assume that if magnet-based school choice creates the 
kinds of market forces envisioned by advocates of choice, these forces 
should produce observable achievement differences adjusting for demo- 
graphic characteristics of the student populations served. 

As in all research of this type, the demographic variables we used in 
our statistical models weredetermined by what was avai I ableinthedata 
sets. At the student level, NAEP does not have a measure of family or 
household i ncome and does not col led free-l unch el igi bi I ity data. Survey 
information collected by NAEP asks students to report the level of 
academic degree of their parent(s)/guardian(s), ranging from some high 
school to graduate degree (PARED). At the school level, NAEP provides 
i nformati on based on census data on the average i ncome of the nei ghbor- 
hood wheretheschool is located (SCH I N C). At thedistrict level , we used 
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percent of ch i I d ren i n poverty (POVERTY) beca use this variableiswidely 
accepted as an appropriate measure of the demographic factors that 
create challenges for school districts' education mission. When people 
describethe population served by a school district as "disadvantaged" or 
"impoverished," it is the proportion of children living in poverty they refer 
to, not so much parents' educational background or how many middleor 
upper income families there are, although it is understood that all these 
variables are correlated with one another. 

Results 

The results are organized with respect to the various school choice 
comparisons. For each comparison, we examine its effect on 4 th grade 
reading achievement, 4 th grade mathematics achievement, and 8 th grade 
mathematics achievement. In what follows, Model 0 refers to the 
unconditional model; Model 1 adds the district choicecomparison at the 
district level; Model 2 adds parental incomeat the student level; model 
3 adds school median income at the school level; Model 4 adds district 
poverty at the district level . 

Magnet v. non-Magnet Districts 

Table5showstheresultofthemulti level analysiscomparing magnet 
districtstonon-magnet districts on 4 th grade NAEP reading achievement. 
M odel 0 di spl ays the breakdown of the percent of vari ance accou nted for 
by differences among students, schools, and districts. It can be seen that 
only about nine percent of the vari ance in 4 th grade reading achievement 
can be accounted for by differences among districts. The majority of the 
vari ance (approximately 73%) can be accou nted for by differences among 
students, with about 19% accounted for by differences among schools. 
M odel 1 di spl ays the M agnet school effect without control I i ngfor parental 
education, school median income, or district poverty. Here we find 
significant differences in favor of non-magnet districts. High magnet 
districts show a greater difference than low magnet districts when 
compared to non-magnet districts. Models 2 through 4 add parental 
education, school median income, and the number of children living 
below poverty in the district, respectively. It can be seen that for each 
additional control variable the differences among magnet and non- 
magnet districts decrease. Substantial decreases are observed when 
adding school median income and district level poverty. It may be 
interestingtonotethat adding parental education does not account for a 
substantial amount of variation in student level achievement, whereas 
the addition of school median income and poverty result in a compara- 
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tively larger decrease in variance explained by schools and districts 
respectively. 

In Model 4, it can be seen that the high magnet district's coefficient 
remains significant, although thesizeof the remaining difference is very 
small (-5. 9 points), adifferenceof about l/7 th of a standard deviation. Thus, 
the mean NAEP scoreof the high magnet districts is slightly lower than 
the mean of thecomparison group, defined asdistricts without magnets 
and school choice. 


Table5. Multilevel Models of the Effect of Magnet Schools 
on NAEP 4 th Grade Reading Achievement 



Model 0 

Model 1 

Model 2 

Model 3 

Model 4 

Fixed Effects 

District Mean 

210.812 

208.916 

208.879 

208.874 

212.235 

Low Mag 


-5.235* 

-5.091* 

-2.542 

-0.786 

High Mag 


-11.239* 

-11.063* 

-8.856* 

-5.895* 

Variance Components 

Student Level 

989.906 

989.922 

984.768 

984.850 

984.835 

School Level 

254.605 

253.551 

253.501 

181.854 

182.184 

District Level 

120.084 

102.913 

103.020 

65.860 

48.378 

Variance Decomposition (Percentage by level) 

Student Level 

72.54 

73.52 

73.42 

79.90 

81.10 

School Level 

18.66 

18.83 

18.90 

14.75 

14.99 

District Level 

8.80 

7.64 

7.68 

5.34 

3.98 


Model 0: Fully unconditional model 

Model 1: School Choice policy added at district level 

Model 2: Model 1 +parental education added at student level 

Model 3: Model 2 + school income added at school level 

Model 4: Model 3 4-district poverty measure added at district level 

* p <.05 


Table 6 shows the results for the multilevel analysis comparing 
magnet districts to non-magnet districts on 4 th and 8 th grade NAEP 
mathematics achievement. Again, it can be seen that individual differ- 
ences among students account for the vast majority of thetotal variation 
in mathematics achievement scores. Differenceamongdistricts account 
for only a small fraction of the variance. I n addition, it can be observed 
that thedifferences among magnet and non-magnet districts shrink after 
accounting for demographic characteristics of students, schools, and 
districts; only the high magnet category retains a statistically significant 
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coefficient, although the effect sizeis very small— a littleover l/8 th of a 
standard deviation. 

The largest decrease in the difference among magnet and non- 
magnet districts occurs after adding district poverty. In terms of ex- 
plained variance, it can be seen that adding school median income and 
district poverty respect i vel y expl a i n su bst a nt i a 1 1 y mor e va r i at i on i n mat h 
achievement compared to adding parental income at the student level. 

Neighborhood v. “Other” Districts 

Table 7 shows the result of the multilevel analysis comparing 
districts with neighborhood assignment polices to districts without 
neighborhood assignment polices on 4 th grade NAEP reading achieve- 
ment. Model 0 displays the breakdown of the percent of variance 
accountedfor by differ ences among students, schools, anddistricts. 1 1 can 
be seen that only a fraction of the variance in 4 th grade reading 
achievement can be accounted for by differences among districts. Model 
1 displays the neighborhood district effect without control I ingfor paren- 
tal education, school median income, or district poverty. Here we find 
significant differences in favor of neighborhood districts. Models 2 
through 4add parental education, school median income, and thenumber 
of chi Idren living below poverty in thedistrict, respectively. It can beseen 
thatforeach additional control variablethedifferencesamong neighbor- 
hood and non-neighborhood districts decrease as does the percent of 
explained variance, with the most dramatic decrease occurring when 
addingdistrict poverty. 

Finally, Table8showstheresultsforthe multilevel analysiscompar- 
ing neighborhood districts to non-neighborhood districts on 4 th and 8 th 
grade NAEP mathematics achievement. As with the previous analyses, 
differences among districts account for only a fraction of the variance. I n 
addition, itcan beobservedthatthedifferencesamongneighborhoodand 
non-neighborhood districts decrease after accounting for demographic 
characteristics of students, schools, and districts. I n addition, as with 
reading, thelargest decrease in the difference among neighborhood and 
non-neighborhood districts as well as percent of variance explained 
occurs after adding district poverty. 

“Other” Choice v. Magnets Or No School Choice Districts 

Across both grade levels and for both reading and mathematics 
achievement, there were no significant differences in mean district 
achievement scores among districts that reported having school choice 
and those that did not. Therefore, there was no need to control for 
parental education, school median income, or district level poverty. 
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Table7. Multilevel Models of the Effect of Neighborhood Schools 
on NAEP 4 th Grade Reading Achievement 


Model 0 

Model 1 

Model 2 

Model 3 

Model 4 

Fixed Effects 

Average District Mean 

210.771 

208.916 

208.879 

208.874 

210.510 

Neighborhood Schools 


6.013* 

6.094* 

6.093* 

1.183 

Variance Components 

Student Level 

995.957 

995.960 

991.396 

991.358 

991.477 

School Level 

255.032 

254.938 

254.441 

254.564 

182.441 

District Level 

118.969 

111.530 

112.115 

112.068 

53.486 

Variance Decomposition (Percentage by level) 

Student Level 

72.70 

73.10 

73.01 

73.00 

80.78 

School Level 

18.62 

18.71 

18.74 

18.75 

14.86 

District Level 

8.68 

8.48 

8.26 

8.25 

4.17 


Model 0: Fully unconditional model 

Model 1: School Choice policy added at district level 

Model 2: Model 1 + parental education added at student level 

Model 3: Model 2 + school income added at school level 

Model 4: Model 3 +district poverty measure added at district level 

* p <.05 


Adding control variables either did not change the mean differences 
appreciably or the mean differences became even smaller. 

Discussion 

Our findings are inconsistent with optimistic claims about the 
possi bi I ity of magnet-based school choice engenderi ng di strict -wide pro- 
ductivity gains through forces of parent choice and school competition; 
our findings are more consistent with views that, whatever salutary 
benefits i ndi vidual magnet school s may have, there is no reason toexpect 
t h at systemi ca 1 1 y magn et/ch oi ce d i st r i cts a re very d i fferent f rom d i st r i cts 
without magnet schools and with student assignment policies linked 
more with zoned attendance areas. 

None of our statistical models produce evidence of a strong associa- 
tion between district-level student assignment systems and student 
achievement measured on NAEP in reading and mathematics. I ndeed, 
theonly statistically significant associations uncovered indicated a small 
differ encein NAEP scoresfavoringthenonchoicedistricts compared with 


Table 8. M ulti level Models of the Effect of Neighborhood Schools 
on NAEP 4 th and 8 th Grade Mathematics Achievement 


Douglas A. Archbald & David Kaplan 


25 



CO 


rH 

P 

(N 


CO 

p 

CM 

CO 

rH 

03 

i— 1 

LO 

rH 


O 

P> 

o 

o 

LO 

ro 


sz 

-M 

03 


Tf 

i— 1 

03 


p 

03 

d 

i— i 

ro 

LO 


•Tf 

CO 

o 

i— i 

d 

Tf 

Z 


Psl 

o 


03 

i— i 






aj 

E 



PM 

PM 

LO 

PM 


Ln 

cm 

p 

ro 

p 

(N 


Ln 

p>- 

ro 

i— i 

rsj 

i — i 

s: 

sz 


PM 


o 

in 

CO 


■ — i 

ro 

d 


03 


Tf 

PM 


03 

ro 


CO 

i— i 



Z 


pm 

pm 

o 


LO 

p 

rsi 

i— i 







CO 


p 

^ . 


LO 

ro 

P 

ro 


p- 

Ti- 

03 



00 

i — i 


LO 

o 


rsj 

ro 

rsi 


sz 


Ln 

rsi 


rsi 

oq 



■ — i 

03 

03 

m 

"ro 


03 



p 

d 



CO 





m 

rsi 

Tf" 


03 

03 

i— i 
i — i 

rH 





E 



rH 



Ln 

o 

03 


rsi 

ro 

o 




LO 



rsi 

Ln 

03 


rsi 

p 

i— i 


sz 

4— < 


PM 

03 


o 

LO 

O 


03 

rsj 

CO 


03 


ro 

CM 


03 

rsi 

03 


p* 

i— i 



Z 


rH 

CM 

rsi 


LO 

p 

rsi 

rH 

P* 














aj 













> 













0) 





£ 


ro 



Ln 

ro 

rsi 

03 

LO 


Psl 

tT 

Tf 


CO 

SZ 


03 

o 

p 

p 


i — i 

ro 

00 

LO 

* 

LO 

LO 

tJ; 
i — i 

p 

fN 

rsj 

ID 

4-1 

03 

Z 


00 

m 

pm 

P 

d 

to 

p 

03 

03 

o 

Ln 

i— i 

O 

P~ 

i — 1 

0) 

03 

03 

P' 

i— i 

i— 1 

E 

s: 

M ath 4 th 

to 

13 

y= 

LO 
03 
i — i 
i — i 

* 

00 

ro 

03 

4 -1 

c 

a; 

c 

o 

Q. 

E 

p 

Tf 

03 

CO 

LO 

psl 

Tj- 

o 

p 

LO 

03 

in 

p^ 

rsi 

4—> 

6 

U 

b 

Q_ 

71.82 

16.25 

11.92 


LU 

CM 


o 

u 

p 

i— i 

rH 

C 

o 






"8 



a; 




4-1 






X 



U 



Ln 

03 

IS) 





CO 

Ll_ 

ro 

03 

o 

* 

c 

03 

p 

p 

CO 

03 

o 

CL 

03 

in 

5 

P 

03 


sz 


ro 

Q 


p 

rsi 


E 

LO 

i — i 

rsj 

rH 

4— 1 
03 

Z 


00 

Ln 

PM 

CO 

d 

03 

> 

03 

03 

03 

i — i 
Ln 

i— i 

p> 

rH 

o 

b 

Q 

p- 

i— i 

rH 

aj 

E 

x: 





Tf 

03 

O 

a; 

u 

Ln 

p 

00 

Tf 


rn 

03 



Tf 

i — 1 

LO 

c 

03 

rH 

00 

s: 

SZ 


i — i 


03 

i— 1 

03 

03 

i — i 

d 

i — i 


4-1 



o 


•sf 

•rf 

P^ 

’i_ 

Pn 

i— 1 

i — i 


03 


j~j 

d 


p 

p 

rsj 

03 





Z 


r\i 


p 

rH 

rH 

> 





CO 


PM 

CO 

ro 



p 

p 

rsj 

s 

rH 

ro 

Ln 


LO 

o 

CO 

ro 

Ln 

Ln 


sz 

4-1 

03 


o 

LO 



03 

03 

i — i 
Ln 

o 

CO 


LO 

P- 

i — i 

i — i 

ro 

i— i 

o 

Z 


PM 



03 

i— i 

i— i 





TD 













E 

z 

Tf 

sz 


O 

03 

•Ti- 



LO 

tT 

03 

p 

m 

P 

rH 

m 

p** 


Psl 

in 

Tf 

o 

tT 

tT 


4-> 

03 


ro 

rH 



•sf 

p 

ro 

p 

ro 


i — i 
p- 

d 

i — i 

rsj 

i— i 


Z 


PM 



p 

i — i 

rH 








C 

ffi 

13 

tfc 


ID 

> 

aj 

N 

aj 


aj 

5 

aj 

aj 

> 




<D 

CD 


d) 




> 








_i 

s 

_i 


_i 

aj 

_i 




t> 



4-1 

C 

_l 

5 


4-1 

c 

8 

tj 

tj 

a; 



"i_ 

4-1 

sz 


a; 

■a 

8 

\— 

4-> 


a; 

~o 

4-> 

4— 



IS) 

sz 


o 

-6 

LO 

IS) 


o 

SZ 

u 

LO 

1/3 

LU 



Q 

2 


LO 

Q 


tn 

Q 


u 

8 


■jj 

s 


If) 


C Tfj "O 

> 4_, 

03 

tn -g "S 

u OT) 
rn si ~U 
u 03 
<U 


■cTB 

* 5 

■° -g ra 

■>=; c to ® 

”.??E 
E fq ? >■ 

■D U 

-O =3 £ Jj 

^E 8 8 
q.S'is = tt 
5 o| 

■J=i (D O J- 


CU 

"g: 

E 


C O Q- 1/1 T3 
8 ° + + + 
3 U H (N m 
>,Q ai aj aj 

Li- LD ^ ^ Z 


o«-ifNjm'sj- 

aj aj aj aj aj 

E E E E E 


26 


Parent Choice Versus Attendance Area Assignment to Schools 


just the high magnet districts (but nodifference when compared with the 
magnet districts with a lower prevalence of magnet schools). Wesuggest 
caution in interpreting the substantive significance of the small, but 
stati stical ly signifi cant differences between thecategories of districts for 
several reasons: first, the large sample size (approximately 30,000 
students) allows small differences to achieve statistical significance; 
second, it isknownfrom other research studiesthat districts implement- 
ing magnet schools have histories of racial segregation and on average 
are more I i kel y to be di st ri cts encompassi ng I arge nu mbers of i n ner city 
schools- schoolsthat mayappeardemographicallysimilaron available 
measures but that in reality often faceextraordi nary challenges of family 
dislocation, student residential mobility, and neighborhood drugs and 
violence 0 argowsky, 1996; Wilson, 1997); and third, much research 
shows that magnet schools themselves, if not their districts asa whole, 
tend to have relatively high performance. 

Bel ow, we present fi rst, two i nterpretati ons of these results we cal I , 
respectively, "zero-sum reallocation" and "adverse reallocation. "Then, 
because some research indicates magnet schools often have superior 
academic outcomes, we offer two interpretations that may reconcile our 
findings with this previous research. 

Zero-sum reallocation suggests that the kinds of school choice 
policies examined in this study essentially re-allocate students among 
schools, but generate no endogenous productive forces. That is, magnet 
schools and the school choice policies in which magnets operate may 
createchoices and educational opt ions for fa mi lies, and even have other 
benefits, but they do not create market forces making schools better in 
ways that lead to higher academic achievement. School choice, in this 
view, is largely a processofredistributing students among schools within 
the district. There may be reasons to do this and benefits from this, but 
this process does not affect achievement on a district-wide level enough 
tobedetectablebyNAEP assessments of reading and mathematics when 
compared with districts that assign students totheir schools, with little 
or no family choice involved. (This view is based on the cautious 
interpretation of our results discussed earlier that our findings do not 
produce evidence to i nfer i mportant substantivedifferences between the 
categories of districts as measured by N AE P scores.) 

Is zero-sum reallocation inconsistent with other research indicating 
magnet schools have higher achievement scores? 1 1 is not i nconsi stent if 
magnet schools' higher achievement scores result from self-selection 
processes withi n districts. Assumi ng a constant composition of students 
withinthedistrict,17thenthemovementof higher-scoringstudentsfrom 
other schools to magnets raises magnet schools' test scores at the 
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expense of other schools. This sort of reallocation will not raise the 
district average. If school choice— at least the forms of public school 
choice examined here— does not do much morethan real locate students 
among schools, then there is I ittle reason to expect average scores among 
school choice districts to be higher than average scores among districts 
without school choice policies. 

A second more critical interpretation— adverse reallocation— sug- 
gests magnets may actually have adverse effects within a district. This 
i nter pretat i on wou I d focus on t he smal I negat i ve coeff i ci ents of t he h i gh 
magnet category of districts. Some argue magnet schools may have a 
detri mental effect on non magnet schools by drawingoff the best students, 
in effect depleting non magnets of model students that would otherwise 
bepresent tothe benefit of less ablestudents;somealso contend magnets 
relegate nonmagnets to a "second class" status, with adverse morale 
consequencesfor staff moraleand parent support (Bastian, 1985; Carrison, 
1981; Moore and Davenport, 1989; Pearson, 1993). 

On the other hand, if we assume that magnet schools do indeed 
produce higher achievement among their students and we assumethat 
this does not come at the expense of nonmagnet schools, 18 then 
districts with a sizable number of magnet schools should have higher 
average achievement than districts without magnet schools. Perhaps 
this is so, and NAEP achievement tests are not sensitive enough to 
measure differences in achievement between students in the two 
different pol icy contexts (magnets/school choice versus no magnets/no 
school choice). I n our sample, of those districts with magnet schools, 
three-fourths had one-third or less of their schools as magnet schools; 
half had 12.5%or I ess of their schools as magnets. Thus, even if magnet 
school s' achievement isgreater, at a district level thedifferencemay be 
insufficient to boost NAEP reading and math means intheschool choice 
category (relative to the means of the comparison groups) by statisti- 
cal ly signifi cant margins. 

A second possi bi I i ty i s that h i gher ach i evement from magnets/school 
choiceisin knowledge or skill areas different from those areas assessed 
by NAEP mathematics and reading tests. As described earlier, magnet 
sch ool s i n t he great maj or i ty of cases h a ve "sped a I ty " a reas— d i st i net i ve 
pedagogies or curricular foci (Metz, 1986; Steel & Levine, 1994). Conceiv- 
ably, achi evement differ ences disti nguishi ng magnet schoolsfrom other 
schools without these specialty areas are not effectively measured with 
conventional standardized tests of reading and mathematics. Thus, there 
may indeed be differences— but NAEP does not measurethem. 

These alternative scenarios should guide future research. The 
severe paucity of research exploring links between district or state level 
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school choi ce policy and academic outcomes allows a number of plausi ble 
i nterpretati ons of our fi ndi ngs. 

1 1 i s i mportant to proper I y del i mi 1 1 h i s st udy to for ms of pu bl i c school 
choice policy associated with magnet schools. There are a growing 
number of approaches to school choice, many more now than there were 
ten years ago. Accordingtosome,the"publicschool only” choice models— 
particularly the kind examined in this study— are not system-changing 
reforms (Chubb & Moe, 1990; Erickson, 1982; Everhart, 1982; Friedman 
& Friedman, 1981; Lieberman, 1986; Flarmer, 1994, Peterson, 1990). 
They do not create market forces powerful enough to change the 
operation and effectiveness of public schools. According tothis view, true 
education markets can only occur if therearemany schools within a given 
area that are not managed by the traditional public education bureau- 
cracy, these schools are accessible and known to parents, school perfor- 
manceiseffectively measured and publicly reported, and all schools have 
i ncent i ves to att r act a n d sat i sfy t hei r cu stomer s a n d di si n cent i ves to I ose 
or dissatisfy customers. Thereisgrowingsupport for tuition voucher and 
charter school policies becausetheymoreclosely satisfy theseconditions. 

Finally, a methodological note is warranted concerning assumptions 
about the classification of the school districts in the sample based on the 
tel ephone interview data. For analytic purposes, thedistrictsinthesample 
are placed in various dichotomous categories. We realize this is a simpli- 
fication and recognizethat school choice in practice is a complex operation 
and in practice districts do not fall neatly into dichotomous categories in 
thei r approaches to assigni ng students to school s. There are di stri cts that 
would say "no, we do not have a school choice policy" that in fact may take 
a fairly liberal approach in allowing student requests to transfer among 
schools; andtherearedistrictsthat wouldanswer "yes"tothe presence of 
magnet schools and school choi cethat upon closeinspection mayarguably 
have a fai rly restricted approach toschool choice. Whi lethere is no reason 
tosuspectsystematicmisreportingamongthetelephoneinterview respon- 
dents, we would be remiss not to acknowledge the simplification that is 
i mposed by pi aci ng di stricts i n a few broad categori es. 1 1 wou I d be i deal to 
have a more comprehensive base of information on student assignment 
policiesandpracticesfor a large sampleof districts, enabling more refined 
cl assi f i cati ons al ong a conti nuumranging from central i zed assi gn ment to 
progressi vel y greater degrees of school choi ce. 19 T h i s wou I d be a product i ve 
direction for future research. 
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ment of Education, OERI National Institute on Educational Governance, Fi- 
nance, Policy-MakingandManagement.Opinionsreflectthoseoftheauthorsand 
do not necessari I y refl ect those of the granti ng agency. 

1 For more on these themes, see the "M agnet Schools of America" website: 
http://www.magnet.edu/ 

2 Ibid. 

3 "Selective" means based in teacher recommendations and in some cases 
GPA or test score-based admissions criteria. This study used tests from the 
Second International Mathematics/Science Study rather than "off-the-shelf" 
standardized tests because they are more advanced, and were more suitable for 
the sample in this study, and because they are considered to be high quality 
academic tests. For more information see Science Achievement in Seventeen 
Countries: A Preliminary Report (NewYork: Pergamon Press, 1988). 

4 For instance, parents choosing magnet schools tend to be more informed 
about educational options in their district and have higher educational attain- 
ment. 1 1 i s possi bl ethat magnet students' h i gher ach i evement owes i n part or even 
entirely to their parents' and their own motivational or academic ability traits, 
and nottothedifferential effectiveness of magnet schools. Although somestudies 
haveemployed sophisticated controlsfor family background and/or student prior 
achievement, selection bias remains a concern. 

5 Gamoran (1996)fu rther assessed theeffects of non-independenceof students 
within schools utilizing a multi level analysis and, in addition, examined theeffect 
of selection bias resulting from non-random assignment to these various schools. 
Although there were several methodological issues that raise some cautions, the 
mu Iti I evel anal yses further supported the positive effect of attendi ng magnet. T o 
control for selection, Gamoran specified a FI eckman-type two-stage model where, 
in the first stage, a selection model was specified as a function of eighth grade 
achievement, sex, race/ethnicity, SES, and family composition plus a variety of 
measures that Gamoran argued tapped into reasons why parents might choose a 
magnet school unrelatedtoachievement advantages. I nthesecond stage, Gamoran 
incorporated theprobability of attendi nga magnet school derived from theselection 
model intotheregression model for achievement and obtained adjusted estimates 
oftheeffect of school typeon achievement, adjusted for thenon-random assignment 
of studentstoschools. A I imitation of Gamoran'sstudyisthathedid not incorporate 
theselection model into the multilevel analysisof the effects of choice on student 
achievement. Thus, whileGamoran correctly recognized the problems associated 
with clustered student data and non-random selection, hedid not combinethetwo 
methods so as to control for both problems simultaneously. 

6 For a review of these and other empirical studies of school choice see 
Goldhaber, 1999. 

7 Flowell et al. (2001) respond to these methodological issues. 

8 This system helps insure valid coding of data. The system automatically 
performed "a 1 1 owed- value" checks and would not permit entry of out-of-range 
data. For example, "yes/no" items wererestrictedtoyesor noresponses (including 
"don't know," "refused, ""inapplicable"); numeric responses were restricted to a 
reasonabl e range of val ues. 
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9 This was known from other policy information sources (e.g., compliance 
recordsfromtheOfficeof Civil Rightsaswell asfundingand program information 
managed by the federal Magnet School Assistance Program) and from previous 
research (e.g., Blank, Dentler, Baltzell, & Chabotar, 1983). 

“For details of thesamplingframe, see Appendix A (Steele& Levine, 1994). 

11 School-level free-lunch data for a number of school districts were missing 
in the CCD data set for the years we needed. For these districts, we contacted 
directlytheindividualsresponsiblefor managingthedistrict free-lunch data and 
obtained in many cases the needed school-level free-lunch counts. Datafieldsfor 
districts without school-level free-lunch figures were coded as "missing" and not 
computed in our analyses. 

12 The Emergency School Aid Act passed in 1975 and subsequent legislation 
(Education for Economic Security Act, Magnet Schools Assistance Program) 
hel ped support thedevel opment and i mplementati on of magnet school programs. 
For more on this, see Glenn (1978), and Warren (1978). 

13 For more details on this, see the "Data Compendi urns" for NAEP 1992 
reading and mathematics available from NCES. 

14 "A magnet school is defined as a public school whose primary purpose is 
to meet all three [emphasis original] of the following goals: to offer a special 
cu rr i cu I ar theme or method of i nstr ucti on , such as math/sci ence, performi ng arts, 
or open classroom; toattract at least some students voluntarily from outsidean 
assigned neighborhood attendance zone; to improve desegregation by meeting 
specific race/ethnic goals (i.e., it must have some type of race/ethnic controls or 
targets." Respondents werefreetoask questionsoftheinterviewer for additional 
clarification of terms. 

15 We conducted T-tests on selected sets of means and found no statistically 
significant differences (P =.05). Mag v Nomag on free-lunch eligible percentages 
& percentage black students. 

16 Multilevel modeling requires that a choice be made with regard to the 
centering of predictor variables. The issue of centering concerns a rescaling of 
predictor variables for the purposes of providing interpretable intercepts of a 
linear model. In standard linear regression, centering is usually not an issue 
because the intercepts are typically not quantities of substantive interest. I n 
multilevel regression however, interceptsarevery important quantitiesreflect- 
ing variability in the average values of within group variables across groups. The 
choice of centering determines the interpretation of the i ntercepts. (SeeBryk & 
Raudenbush, 1992 pp 26-27 for details.) 

17 Some evidence, however, indicates magnets may actually retain or attract 
students who otherwise would not be enrolled in the district - they would 
otherwise be in a private school or attend a school outside the urban system in 
which magnets ordinarily exist. 

18 Several of thestudies of magnet schools presented earl ier used controlsfor 
self-selection processesand produced evidencethat magnets' higher achievement 
is not merely a selection phenomenon, but a result of processes endogenous to 
magnet schools. 

19 SeeArchbald (1996) for more theoretical and methodological elaboration 
on this idea. 
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Appendix 1 

We employ a three-level hierarchical linear model with randomly varying 
intercepts. For thepurposesofthisstudy, interest centered on model i ng changes 
i n a verageach i evement (as esti mated by the i ntercept of the model ) as a f u net i on 
of school choice indicators after controlling for student, school, and district 
characteristics. The relationships between these different demograph ic charac- 
ter stiesand achievement arenotthefocusof this study. Therefor e,theflexibi I ity 
of hierarchical linear model ingal lowsustofocuson changes in theintercept while 
allowing slope variability without attempting to model that variability. 

Wetestedthenormalityofthedistributionsof NAEP'splausiblevaluescores 
for 4th grade reading and math (1992) and for 8th grade math (1994). I n each 
instance the distributions were highly normal as evidence by skewness scores 
averaging -0.19 and kurtosis scores avering -0.10. 

The specification of the three-level model is as follows. The level lmodelcanbe 
written as 

Level-1: Ac\ k ojk ljk ( PARED) ijk e ijk , (Al) 

whereAch ijk istherelevant NAEP achievement scorefor student i, in school j, in 
district kand PARED is the measure of parental educati on for student i, in school 
j, in district k. Assuming the PARED is grand-mean centered, p 0jk is the adjusted 
mean achievement scores that are assumed to vary across schools and across 
districts, p ljk is thesloperepresentingtherelationship between student achieve- 
ment and student PARED, and q jk i s a random error term. 

The Level-2 model can be written as 

Level-2: ojk ok lk (SCHINC) jk r jk , (A2) 

where b 0k i stheadj usted school mean achi evement scores, adj usted for thepercent 
of students whoarefreelunch eligible, b lk isthesloperepresentingtherelation- 
sh ip between school meanachievementschool median income, and r jk isa random 
error term. 

The Level-3 model can be written as 

Level -3: 

O* coo oor( POVERTY ) k ^(CHOICE), u ok , (A3) 

whereg 000 isthegrand-mean achievement score, g 001 isthesloperepresentingthe 
relationship between district level poverty and district level student achieve- 
ment, g 002 isthesloperepresentingtheschool choice comparison of interest, and 
u ok is a random error term. I n the level -3 model, grand mean centering is not 
employed for CHOICE. Estimation of the model parameters utilizes full- 
information maximum likelihood estimation of the variance components along 
with empirical Bayes estimation ofthefixed effects of the model (Raudenbush 
and Bryk, 2002). 


